This invention relates to testing an interactive voice response system and in particular to bulk testing of voice messaging applications.
A function of an interactive voice response (IVR) system is to provide an automated way of communicating information between a user and a business. Although it is unlikely that automated IVR systems will totally replace human operators in the near future they are becoming more pervasive throughout business for simple interactions. For instance while it is still the norm to deal with a human for directory enquiries for national and international calls, simple introductory functions such as acquiring a name are now performed by an IVR and a human interjects with the response or to make a further query. However a fully automatic company IVR directory application exists for company wide enquiries that can respond and query further when there is a problem. It can not be long before these fully automated directory enquiry applications become more pervasive for national and international use.
Voice mail is an IVR application whereby a caller can leave a message for a receiver by dialling directly for connection into a voice mail server. The caller can indirectly leave a message after being connected to a voice mail server after an engaged or timed out telephone call to a party. Once connected to the voice mail server an IVR voice mail application is initiated to service the call. A simple IVR voice mail application may begin by playing a voice prompt asking for the destination voice mail box number (if it does not already know it) and then ask the caller to leave a message which is recorded and stored in the voice mail box. A more complex IVR voice mail application will give the caller various options, for instance, to delete the message, to re-record the message or to forward the message to other voice mail box owners. If the called party who owns the voice mail box wants to listen to the voice mail, a connection to the voice mail server must be made whereby a different voice mail application is initiated. A simple application might allow the owner to listen and delete his messages but a more complex one might allow saving, replying, forwarding to another or other voice mail boxes, or even saving as an attachment and forwarding as an email to somewhere.
During development of a voice response system application, such as a voice mail application, it is necessary to simulate a plurality of calls so that the performance under strain can be monitored. Such a simulation can be performed by a bulk call generator which makes telephone calls to the IVR directly or through a private branch switch. Each simulated telephone call is programmed into the bulk call generator and associated with a particular application. Several types of telephone call may be designed and the bulk caller generator may simulate one or a combination of different calls at the same time using several channels connected to the IVR system.
However some problems still occur in the application which performance testing using programmed simulated calls does not find. This is because real callers behave in unpredictable ways which were not expected or assumed by the programmers of the simulated calls. Functional and/or performance tests are known to be artificialxe2x80x94and although the programmers of the tests may attempt to simulate various aspects of real-life use of voice response systems this is often done by the tester""s inside knowledge of the code, which adds to the artificial nature of the testing. Customers sometimes report problems occurring which are so difficult to recreate in the laboratory that debugging the problems becomes almost impossible and occasionally the laboratory may even disbelieve the problem occurs at all. Therefore there is a need for a new type of voice application test system which does not rely on pre-programmed telephone calls to test an IVR system but is concerned more with realistic user interactions.
U.S. Pat. No. 5,222,083 discloses a method for use in developing, testing, debugging and trouble shooting a call progress monitoring (CPM) detection system which is utilised, for example, by a voice messaging system (VMS) One embodiment includes the steps of (a) monitoring at least one voice channel to obtain CPM tones and having a CPM detection system associated with the VMS perform an analysis of the CPM tones, the output of which analysis is referred to as CPM Trace Variables; (b) storing the CPM tones and the CPM Trace variables in a trace file; (c) obtaining a first voice channel; and (d) feeding the voice portion of the CPM trace file to a second channel for playing to the first voice channel. This system is for debugging problems with a call progress monitoring detection system and is primarily interested in recording call progress tones (e.g., ring tone, busy tone), in audio format, for outgoing calls and xe2x80x9ctrace variablesxe2x80x9d (analysed call progress tone details)xe2x80x94then xe2x80x9clooping backxe2x80x9d one voice channel to another so that the first channel can simulate the call progress tones originally detected by playing back the original audio recordings. It does not concern itself with recording user interactions, audio or otherwise.
According to one aspect of the invention there is provided a test system as described in claim 1.
This solution provides actual customer data for recreating customer reported problems in the laboratory and for providing performance tests with realistic data. This solution provides an option to record the entire interaction of callers with the IVR system including incoming call data, DTMF keys pressed during the call, voice/silence detection data and hang-up (with exact timings) in a file (or database) for xe2x80x9cplaybackxe2x80x9d on voice response units in the laboratory.
In a preferred embodiment the amount of data recorded by the solution is less than the amount of information in the call as only certain events are logged. Some privacy issues (such as recording peoples"" private messages) also do not complicate the implementation of this solution. If there is a requirement to not record DTMF data in certain circumstances (e.g., passwords, credit card numbers) then appropriate implementation of the solution (i.e., flagging that confidential data is about to occur on a channel, after which the system only records the timing of the DTMF presses, flagging that confidential data has finished, after which the system records all DTMF presses again) could ensure that such DTMF data is not recorded.
One difference between the prior art and the above solution is that the solution records details (not an audio recording) of user interactions such as DTMF input and voice input (i.e., the fact that voice input is occurring, not the audio of the voice input nor any details of what is actually being said) for incoming calls. The solution is not concerned about network interactions and the subsequent success or failure of outgoing calls, call progress tones, etc. as described in the US patent publication mentioned above.
Advantageously the user interaction data may be extracted directly from the voice input channels during a live user interaction. This might be used for instance during laboratory development whereby the testers are creating telephone calls to be simulated by making telephone calls and whereby the voice response test system records the interactions for later simulation. More advantageously it may be extracted from trace data recorded during real live user interactions. This might be used in the case where trace data is already available from a voice response application.
Preferably the user interaction data may comprise at least one event with at least one associated property, whereby the event may be an indication of a tone event such as when a DTMF key is pressed during the IVR dialogue. Such interaction data may further comprise properties of tone type (such as the specific DTMF key) produced during the call, timing of the tone, and possibly other details such as the duration of the tone and the channel. Alternatively or additionally the event may comprise an indication of voiced or silence event periods with timing and duration properties. In a further alternative or addition the event may comprise actual voice data of a voiced period with timing and duration periods. In yet a further alternative or addition the event may comprise an indication of a hang up event and the timing of such an event when it occurs in a voice response application.
The voice application test system preferably further comprises backup means for recording the original state of the first telephony application including associated data, variables and registers. This means that diagnostic tests will be more realistic since traffic and specific problems can be recreated in the laboratory based on data of actual user interactions rather than requiring the customer to recreate the traffic or specific problem for the laboratory. The data could include as much detail as all of the voice prompts and the entire contents of the customer database.
Additionally the voice application test system preferably further comprises a call editing tool so that the user interactions as recorded by the extracting means may be manipulated and changed. In the case of performance tests, once the data had been recorded it could be manipulated in ways which should preserve the original xe2x80x9cmeaningxe2x80x9d of the data (e.g., the time between calls could be decreased as long as the order of the calls was maintained).
The extracting means preferably identifies and records user interactions from more than one voice response application dialogue and stores user interactions from each dialogue in a separate and corresponding file. However in an alternative embodiment the extracting means could store each user interaction for all the dialogues in a single file as long as the channel number was associated with the calls so that the appropriate channel could be used when reproducing each user interaction.
The extracting means associates each file with properties of the dialogue so that each call may be uniquely identified and selected by the simulation means. Most preferably the extracting means associates each file with the absolute time it started recording the user interactions for that dialogue so that the simulation means selects each file in the order in which the actual interaction occurred.